Telegram Group & Telegram Channel
πŸ–₯ Phi-3-mini Π² 30 строках Π½Π° C# с ONNX Runtime GenAI

Π’ Ρ€Π°ΠΌΠΊΠ°Ρ… запуска Phi-3 Microsoft выпустила ΠΎΠΏΡ‚ΠΈΠΌΠΈΠ·ΠΈΡ€ΠΎΠ²Π°Π½Π½Ρ‹Π΅ ΠΌΠΎΠ΄Π΅Π»ΠΈ ONNX, ΠΊΠ°ΠΊ ΠΏΠΎΠ΄Ρ€ΠΎΠ±Π½ΠΎ описано Π² ΡΡ‚Π°Ρ‚ΡŒΠ΅ Β«ONNX Runtime supports Phi-3 mini models across platforms and devicesΒ».
Π’Π°ΠΊΠΆΠ΅ ΠΌΠΎΠ΄Π΅Π»ΠΈ Phi-3 mini ΠΎΠΏΡƒΠ±Π»ΠΈΠΊΠΎΠ²Π°Π½Ρ‹ Π½Π° HuggingFace πŸ€—

Π˜ΡΠΏΠΎΠ»ΡŒΠ·ΡƒΡ всё это, ΠΌΠΎΠΆΠ½ΠΎ Π»Π΅Π³ΠΊΠΎ Π·Π°ΠΏΡƒΡΡ‚ΠΈΡ‚ΡŒ модСль локально всСго Π² Π½Π΅ΡΠΊΠΎΠ»ΡŒΠΊΠΈΡ… строках C#, ΠΊΠ°ΠΊ ΠΏΠΎΠΊΠ°Π·Π°Π½ΠΎ Π² этом Π³Π°ΠΉΠ΄Π΅.

▢️ Π“Π°ΠΉΠ΄

А Π²ΠΎΡ‚ Ρ‚Π΅ самыС 30 строк:

using Microsoft.ML.OnnxRuntimeGenAI;
var modelDirectory = args.Length == 2 ? args[1] :
@"C:\git\oss\Phi-3-mini-4k-instruct-onnx\cuda\cuda-int4-rtn-block-32";
using var model = new Model(modelDirectory);
using var tokenizer = new Tokenizer(model);
while (true)
{
Console.Write("Prompt: ");
var line = Console.ReadLine();
if (line == null) { continue; }

using var tokens = tokenizer.Encode(line);

using var generatorParams = new GeneratorParams(model);
generatorParams.SetSearchOption("max_length", 2048);
generatorParams.SetInputSequences(tokens);

using var generator = new Generator(model, generatorParams);

while (!generator.IsDone())
{
generator.ComputeLogits();
generator.GenerateNextToken();
var outputTokens = generator.GetSequence(0);
var newToken = outputTokens.Slice(outputTokens.Length - 1, 1);
var output = tokenizer.Decode(newToken);
Console.Write(output);
}
Console.WriteLine();
}


@csharp_1001_notes
Please open Telegram to view this post
VIEW IN TELEGRAM



tg-me.com/csharp_1001_notes/456
Create:
Last Update:

πŸ–₯ Phi-3-mini Π² 30 строках Π½Π° C# с ONNX Runtime GenAI

Π’ Ρ€Π°ΠΌΠΊΠ°Ρ… запуска Phi-3 Microsoft выпустила ΠΎΠΏΡ‚ΠΈΠΌΠΈΠ·ΠΈΡ€ΠΎΠ²Π°Π½Π½Ρ‹Π΅ ΠΌΠΎΠ΄Π΅Π»ΠΈ ONNX, ΠΊΠ°ΠΊ ΠΏΠΎΠ΄Ρ€ΠΎΠ±Π½ΠΎ описано Π² ΡΡ‚Π°Ρ‚ΡŒΠ΅ Β«ONNX Runtime supports Phi-3 mini models across platforms and devicesΒ».
Π’Π°ΠΊΠΆΠ΅ ΠΌΠΎΠ΄Π΅Π»ΠΈ Phi-3 mini ΠΎΠΏΡƒΠ±Π»ΠΈΠΊΠΎΠ²Π°Π½Ρ‹ Π½Π° HuggingFace πŸ€—

Π˜ΡΠΏΠΎΠ»ΡŒΠ·ΡƒΡ всё это, ΠΌΠΎΠΆΠ½ΠΎ Π»Π΅Π³ΠΊΠΎ Π·Π°ΠΏΡƒΡΡ‚ΠΈΡ‚ΡŒ модСль локально всСго Π² Π½Π΅ΡΠΊΠΎΠ»ΡŒΠΊΠΈΡ… строках C#, ΠΊΠ°ΠΊ ΠΏΠΎΠΊΠ°Π·Π°Π½ΠΎ Π² этом Π³Π°ΠΉΠ΄Π΅.

▢️ Π“Π°ΠΉΠ΄

А Π²ΠΎΡ‚ Ρ‚Π΅ самыС 30 строк:


using Microsoft.ML.OnnxRuntimeGenAI;
var modelDirectory = args.Length == 2 ? args[1] :
@"C:\git\oss\Phi-3-mini-4k-instruct-onnx\cuda\cuda-int4-rtn-block-32";
using var model = new Model(modelDirectory);
using var tokenizer = new Tokenizer(model);
while (true)
{
Console.Write("Prompt: ");
var line = Console.ReadLine();
if (line == null) { continue; }

using var tokens = tokenizer.Encode(line);

using var generatorParams = new GeneratorParams(model);
generatorParams.SetSearchOption("max_length", 2048);
generatorParams.SetInputSequences(tokens);

using var generator = new Generator(model, generatorParams);

while (!generator.IsDone())
{
generator.ComputeLogits();
generator.GenerateNextToken();
var outputTokens = generator.GetSequence(0);
var newToken = outputTokens.Slice(outputTokens.Length - 1, 1);
var output = tokenizer.Decode(newToken);
Console.Write(output);
}
Console.WriteLine();
}


@csharp_1001_notes

BY C# 1001 notes





Share with your friend now:
tg-me.com/csharp_1001_notes/456

View MORE
Open in Telegram


C 1001 notes Telegram | DID YOU KNOW?

Date: |

Telegram Be The Next Best SPAC

I have no inside knowledge of a potential stock listing of the popular anti-Whatsapp messaging app, Telegram. But I know this much, judging by most people I talk to, especially crypto investors, if Telegram ever went public, people would gobble it up. I know I would. I’m waiting for it. So is Sergei Sergienko, who claims he owns $800,000 of Telegram’s pre-initial coin offering (ICO) tokens. β€œIf Telegram does a SPAC IPO, there would be demand for this issue. It would probably outstrip the interest we saw during the ICO. Why? Because as of right now Telegram looks like a liberal application that can accept anyone - right after WhatsApp and others have turn on the censorship,” he says.

What is Telegram?

Telegram is a cloud-based instant messaging service that has been making rounds as a popular option for those who wish to keep their messages secure. Telegram boasts a collection of different features, but it’s best known for its ability to secure messages and media by encrypting them during transit; this prevents third-parties from snooping on messages easily. Let’s take a look at what Telegram can do and why you might want to use it.

C 1001 notes from us


Telegram C# 1001 notes
FROM USA